View-invariant Human Action Recognition Based on Factorization and HMMs

One of the fundamental challenges of human action recognition is accounting for the variability that arises during video capturing. For a specific action class, the 2D observations of different instances might be extremely different due to varying viewpoint when the sequences are captured by moving cameras. The situation is even worse if the actions are executed at different rates. In this paper, a novel view-invariant human action recognition method is proposed based on non-rigid factorization and Hidden Markov Models (HMMs). By assuming that the execution of an action can be approximated by dynamic linear combination of a set of basis shapes, we show that the weight coefficients of basis shapes by measurement matrix non-rigid factorization contain crucial information for action recognition regardless of the viewpoint. Based on the extracted discriminative features, the HMMs is used for action modeling and classification. The performance of the proposed method has been successfully demonstrated experimentally using real sequences.